Collaboration Not Competition

Runing xUnit integration tests in Azure DevOps release pipeline

October 23, 2020

Running unit and integration tests as part of a DevOps pipeline is an important step in spotting deployment issues early and preventing bugs from reaching production. It is relatively easy to run unit tests during the build process but there are a few extra steps required to make sure that integration tests can be run during the release pipeline once deployment to an environment has completed.

Setup

  1. Add an xUnit project to your solution
  2. Write some integration tests - e.g. something that calls a dependency outside of your code like a database or an API
  3. Config Settings - an xUnit project doesn’t have a Startup.cs where we can use the usual .Net Core configuration process. Instead create a static ConfigHelper.cs class that can be referenced in the tests. This also allows us to provide a different environment path for API urls or database connections in each release stage.

public static class ConfigHelper
{
private static readonly IConfigurationRoot Config;
static ConfigHelper()
{
Config = new ConfigurationBuilder()
.SetBasePath(AppContext.BaseDirectory)
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
.Build();
}
public static AppConfig AppConfig
{
get
{
AppConfig appConfig = new AppConfig();
Config.Bind(appConfig);
var env = System.Environment.GetEnvironmentVariable("Environment");
appConfig.APIBaseUrl = env.ToLower() == "prod" ? appConfig.APIBaseUrl.Replace("-[env].", "") : appConfig.APIBaseUrl.Replace("[env]", env);
appConfig.APIKey = System.Environment.GetEnvironmentVariable("APIKey");
return appConfig;
}
}
}
view raw ConfigHelper.cs hosted with ❤ by GitHub

  1. Make sure that the appsettings.json file is copied as part of the build and that it is checked in to source control. This means it is important that secrets are not handled using this approach.
  2. Secrets required for config should be added as Environment Variables. Unfortunately an xUnit project won’t find these in the Visual Studio debug settings so they will need to be added to your System Environment Variables for local debugging. These secrets can be added as Azure Devops release pipeline variables for each release stage.
  3. To ensure that the test project is built and included in the published artifacts and that the integration tests do not run as part of the build add the following steps to the Azure DevOps build:

- task: VSTest@2
inputs:
testAssemblyVer2: |
**\$(BuildConfiguration)\**\*Tests*.dll
!**\$(BuildConfiguration)\**\*Models*.dll
!**\*TestAdapter.dll
!**\obj\**
!**\$(BuildConfiguration)\**\*IntegrationTests*.dll
codeCoverageEnabled: true
testRunTitle: '$(Build.DefinitionName) | $(Build.Reason) |$(Build.SourceVersion)'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
task: DotNetCoreCLI@2
inputs:
command: 'publish'
publishWebProjects: false
projects: '**\MyNamespace.IntegrationTests\*.csproj'
arguments: --output $(Build.ArtifactStagingDirectory)
zipAfterPublish: false
view raw IntegrationTestsBuild.yaml hosted with ❤ by GitHub

  1. In the release pipeline add a VsTest task with the following YAML:

- task: VSTest@2
displayName: 'VsTest - Integration Tests'
inputs:
testAssemblyVer2: |
**\MyNamespace.IntegrationTests.dll
!**\*xunit.runner*.dll
!**\obj\**
codeCoverageEnabled: true
diagnosticsEnabled: true
rerunFailedTests: true
view raw IntegrationTestsRelease.yaml hosted with ❤ by GitHub